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Abstract. In a recent paper, I. Selesnick and C.S. Burrus developed a design 
method for maximally flat FIR low-pass digital filters with reduced group delay. 
Their approach leads to a system of polynomial equations depending on three in- 
teger design parameters K,L,M. In certain cases (their "Region I"), Selesnick and 
Burrus were able to derive solutions using only linear algebra; for the remaining 
cases ("Region II"), they proposed using Grobner bases. This paper introduces a 
different method, based on multipolynomial resultants, for analyzing and solving the 
Selesnick-Burrus design equations. The results of calculations are presented, and 
some patterns concerning the number of solutions as a function of the design param- 
eters are proved. 



§1. Introduction 

In this paper we will present an application of techniques from computational 
commutative algebra and algebraic geometry to a problem in signal processing. We 
will see that recent developments in the theory of multipolynomial resultants give 
a powerful method for solving an interesting family of problems in digital filter 
design. 

We begin by recalling some basic concepts about digital filters. (A good general 
reference for this material is [PM].) A digital signal is a quantized function of a 
discrete variable, (e.g. time). If wc ignore quantization effects, therefore, a signal 
can be represented mathematically by a sequence of complex numbers x[n] indexed 
by n G Z. For many purposes, an appropriate class of signals is the sequence space 
£2, since the finitcness of the £2 norm corresponds to a finite energy condition on 
signals. Signal processing operations can be described mathematically by means 
of operators r : £2 — ► £2- In the signal processing context, these are called digital 
filters. Here, we only consider filters that are linear and shift-invariant: If k is fixed 
and y[n] — x[n + k] for all n, then r(y)[n] = T(x)[n + k]. 

A linear, shift-invariant filter is characterized completely by its transfer function 
H(z), the z-transform of its impulse response (see §1 below). Design methods for 
filters to perform specified operations on signals can often be formulated as finding 



1991 Mathematics Subject Classification. Primary 94A12; Secondary 13P99,68W30. 

1 



2 



SOLVING THE SELESNICK-BURRUS FILTER DESIGN EQUATIONS 



solutions of systems of polynomial equations on the coefficients in transfer func- 
tions H(z) of some specified form. For this reason, techniques from computational 
commutative algebra have begun to find uses in this area. 

In this article we will focus on one particular filter design method introduced by 
Selesnick and Burrus in [SB]. Their idea was to specify H(z) for a low-pass, finite 
impulse response (FIR) filter (see §1) by imposing three types of conditions: 

(1) A given number M of flatness conditions at u = on the square magnitude 
response 

F(u) = \H(e i «)\ 2 

(that is, the vanishing of the derivatives of all orders up to 2M of F(uj) at 
uu = - note that F is an even function so the derivatives of odd orders at 
lu = are zero automatically), 

(2) A second number L of flatness conditions at w = on the group delay 

G(u;) = £argtf(0 

(that is, the vanishing of the derivatives of all orders up to 2L of G(u) at 
cj = - note that G is also an even function of lo), and 

(3) A third number K of zeroes of H{e lul ) at w = w. 

The parameters K,L,M can be specified independently and this approach can be 
seen as a generalization of earlier work on maximally flat filters by Hermann, Baher, 
and others in certain special cases. Each of these types conditions leads to polyno- 
mial equations of degree < 2 on the coefficients h[n] in H(z) — ^^=0 h{n]z~ n 7 and 
solutions exist provided N — 1 > K + L + M. The equations have a particularly 
simple form if the filter moments 



N-l 



TOfc = ^ n k h[n] 



n=0 



are used as the variables. Following Selesnick and Burrus, we express everything in 
terms of the . 

Selesnick and Burrus establish a subdivision of these problems into two classes. 
The easier cases (Region I) occur for L relatively large compared to M : 



M - 1 



< L < M. 



In these cases, Selesnick and Burrus give an analytic solution procedure depend- 
ing only on linear algebra. The more difficult cases (Region II) occur when L is 
relatively small compared to M: 



< L < 



M - 1 



- 1 



In Region II, Selesnick and Burrus used lex Grobner basis computations to solve 
the resulting filter design equations in a few cases. However, the complexity of this 
approach severely limited the range of cases they were able to handle. 
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Some remaining problems left unsolved by Selesnick and Burrus's work are 

(1) to develop an efficient method to solve the filter design equations in the 
Region II cases, and 

(2) to understand the structure of the solutions of the equations for Region II 
in more detail - in particular to determine for given K, L, M, how many 
solutions there are, how many are real, how many yield monotone decreasing 
square magnitude response \H(e luJ )\ 2 , and so forth. 

While we cannot claim a complete solution to these problems, in this article we 
first introduce a different solution strategy for the Selesnick-Burrus equations in 
the Region II cases which has allowed us to compute solutions in cases with much 
larger values of L, M than those reported in [SB]. Our approach is based on a careful 
study of the form of the equations, combined with an application of multipolynomial 
resultants to eliminate variables and obtain a univariate polynomial in the 1st filter 
moment mi. This strategy is laid out in more detail in (3.10) below. (For general 
background on multipolynomial resultants, see [CLO] Chapters 3 and 7, [EM], [S] 
and [CE] for more details on the sparse version, and [KSY] for Dixon resultants. 
[M] contains a number of practical recipes for applying these ideas to solve systems 
of equations.) 

Second, we attempt to explain some of the intriguing patterns we have noticed 
in the solutions, in particular in the number of distinct complex solutions of the 
Selesnick-Burrus equations along the "diagonals" M = 2L + q for various values of 
q. For a given q and L sufficiently large these systems have a similar shape, and 
for the first few values of q giving cases in Region II, we have been able to analyze 
the form of the resultant and determine the degree of the univariate polynomial in 
mi obtained by elimination in all cases. 

The organization of the paper is as follows. §2 contains some additional concepts 
and terminology on digital filters, a presentation of the exact form of the Selesnick- 
Burrus equations from [SB], and a small example (the case K = 1, L = 1, M = 5), 
which illustrates some key features of these problems. In §3, we lay out a successful 
solution strategy for the Region II problems based on resultants. The first step 
consists of two reductions that permit the direct elimination of variables in the full 
Selesnick-Burrus system of K + L + M equations in K + L + M unknowns to yield a 
much more manageable system of M — L — 1 equations in M — L — 1 variables that 
we call the reduced Selesnick-Burrus system. The general strategy is presented, 
followed by some experimental results. 

First, we present an outline of a calculation determining the real solutions of 
the Selesnick-Burrus system with K = 2, L = 2, M = 10, and the square mag- 
nitude response curves of the corresponding filters., For this calculation we use a 
method based on the Dixon resultant, combined with numerical rootfinding. All 
the calculations were carried out in the Maple 8 computer algebra system. 

Second, we give a table showing the number of distinct solutions of the Selesnick- 
Burrus systems for most of the cases with M < 14 in Region II (see Figure 2 below). 
A number of the entries in this table were computed by Robert Lewis of Fordham 
University using his Fermat system and code for Dixon resultants. The resultant 
strategy would allow the computation of many additional cases with M > 15 as 
well. By way of comparison, we note that Selesnick and Burrus were only able to 
handle cases with M < 7 in their paper. 



4 



SOLVING THE SELESNICK-BURRUS FILTER DESIGN EQUATIONS 



In the remainder of the paper we study some of the patterns that are apparent in 
Figure 2. §4 is devoted to a study of the properties of the coefficient matrices of the 
linear parts of the reduced Selesnick-Burrus systems, matrices whose coefficients are 
polynomials in the variable t — mi. By some fairly intricate algebraic maneuvering, 
we are able to express these matrices in a very useful form using some notions from 
the calculus of finite differences. In particular, the entries can be expressed in terms 
of polynomials of the form D J K (i — tY\i=o, where D 3 k are certain finite difference 
operators. This allows us to determine the Smith normal form of these matrices, 
hence to completely understand the dependence of the ranks of various submatrices 
on t. 

The cases with M = 2L + 3 are studied intensively in §5, and the following main 
theorem is established (compare with the data in the table in Figure 2 below). 

(5.1) Theorem. In the cases M = 2L + 3, L > 0, (the "corners" in Region II 
boundary), for all K > 1, the univariate polynomial in t in the elimination ideal of 
the Selesnick-Burrus equations obtained via Strategy (3.10) has degree 8L + 8. 

In §6, we undertake a similar study of the cases with M — 2L + 4 and establish 
our second main theorem. 

(6.1) Theorem. In the cases M = 2L + 4, L > 1, for all K > 1, the univariate 
polynomial in t in the elimination ideal of the Selesnick-Burrus equations obtained 
via Strategy (3.10) has degree 12L + 14. 

The proofs of Theorems (5.1) and (6.1) show in essence how to construct the 
appropriate resultant matrices, so they give a general, extremely efficient, way to 
solve all cases with M = 2L + 3, 2L + 4. Similar results arc possible in principle for 
the lower diagonals M = 2L + q, q > 5 as well. But we will not attempt to prove 
formulas for the number of solutions in those cases here because the resultants 
necessary to handle them become progressively more complicated to analyze. 

In a companion article, [LL], we will discuss the properties of the Selesnick- 
Burrus filters from Region II in more detail. 

The author would like to thank Ivan Selesnick for several valuable conversations, 
and Robert Lewis for permission to present his computational results here. 



(1 at n = 0). 5 is called the unit impulse at n = 0. Let T be a linear, shift- 
invariant filter as in §1. The output T(S) from the filter on input S is called the 
impulse response of T. A beautiful consequence of the linearity and shift invariance 
hypotheses is that the impulse respose of a filter determines the output on any 
other input signal. For, we can write 



§2. Preliminaries on Filter Design 
and the selesnick-burrus equations 



Let (5 be the signal 



0,0, 1,0, (),••■ 




k— — oo 
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If h[n] are the coefficients of the impulse response and and y = T{x) is the output, 
then by linearity and shift-invariance, 

oo 

y [n] = x[k]T{6)[n - k] 

k— — oo 

oo 

= ^ a;[fc]/i[n — k] 

k— — oo 

In other words, the output is the (discrete) convolution of the input and the impulse 
response. 

It is standard in signal processing to package the signals x[n], y[n], h[n] by their 
"^-transforms" X, Y, Z. For instance, the definition of the z-transform of the signal 

x[n] is 

oo 

X(z)= ]T x[n]z-\ 

n— — oo 

The z-transform of the impulse response, H{z), is called the transfer function of 
the filter. In our cases, h[n] will be nonzero for only finitely many n. Such filters are 
called finite impulse response, or FIR filters. For an FIR filter, the transfer function 
is a rational function, hence has a well-defined value at all z in the complex plane, 
except for a pole at z = 0. 

Note that the coefficient of z~ n in the product H(z)X(z) is the discrete convo- 
lution from (2.1) 

oo 

h[k]x[n — k], 

k— — oo 

which is the same as y[n]. In other words, the z-transform of the output is the 
product of the transfer function and the z-transform of the input: Y(z) = H(z)X{z). 
Note that the restriction of H(z) to the unit circle in the complex plane, 

oo 

H{e ibJ ) = h[n]e- inw , 

n— — oo 

is the (discrete-time) Fourier transform of h, so H{z) also determines the frequency 
response characteristics of the filter on input signals. 

Filter design problems, such as the one studied in [SB], ask for constructions of 
filters adapted to perform some specified operation on input signals. An important 
approach is to obtain the desired behavior by designing the form of the transfer 
function H{z). For instance, we might seek to construct: 

(1) "Low-pass" filters in order to remove high-frequency components of signals. 
These typically smooth out or blur signals and can be used to remove high- 
frequency "noise" . 

(2) "High-pass" filters to remove low-frequency components of input signals. 
These typically pick out fine details, or rapid changes in the input and can 
be used to detect features. 



(2.1) 
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The paper of Selesnick and Burrus proposes a way to design maximally flat 
low-pass FIR filters with reduced group delay. These filters are specified by three 
positive integer parameters denoted K, L, M. For an FIR low-pass filter with trans- 
fer function 

JV-l 

H(z) = h[n]z~ n , 

n=0 

let F{lo) be the square magnitude response and G(lo) be the group delay response 
as in §1. Selesnick and Burrus show that if K, L, M E N, and K + L + M + l = N, 
L < M, then the filter coefficients h[n] can be determined to make: 

F( 2i \0) = 0, i = l,...,M, 

(2.2) G^)(0) = 0, j = l,...,L, 

(l + z-') K \H(z). 

The meaning of the first condition is that F(lo) is flat to order 2M at lo = 0. 
Similarly the second equation says G(lo) is flat to order 2L at lo = 0. The final 
equation can also be interpreted as a flatness condition, since it implies that \H{lo)\ 2 
has a zero of order 2K at the normalized frequency lo — it, which corresponds to 
z = — 1 under z = e w . 

It is easy to see that the Selesnick-Burrus conditions (2.2) can be expressed as 
polynomial equations in the filter coefficients. However, the form of these equations 
becomes significantly simpler if they are expressed in terms in terms of the filter 
moments, 

N-l 

(2.3) m k = n k h[n}. 

The explicit form of the equations is derived in [SB] as follows: 

1 . The flatness conditions on F at lo = are quadratic in the m,: 

(2.4a) 0= ^ m 2 + 2j2^y-iy +e rn e m 2l - e , i = l,...,M. 

2. The flatness conditions on G at lo = are also quadratic in the m,: 

(2-46) = E ( X - 2^Tl) I (-^Ww, 3 = 1, • • • , L. 

(These are derived from G^(0) = 0, using the conditions F( 2i) (0) = 0, i = 
1,...,M.) 

3. Finally, the zero of order K at z = — 1 is equivalent to saying that the remainder 
of H (z) on division by (1 + z~ 1 ) K is zero. This yields K linear equations on m^. 

At first glance this looks like an underdetermined system with 2M + 1 variables 
rrii, i = 0, . . . , 2M, and K + L + M = N — 1 equations. However, the moments 
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TOfe, k > N are not independent variables. They can all be expressed in terms of 
to , . . . , rriN-i by solving systems of linear equations. We will normalize our niters 
by requiring that to = 1. This accomplishes a first reduction to a system of N — 1 
equations in N — 1 variables. We expect only finitely many solutions and the real 
solutions are of the greatest interest. 

(2.5) Example. We study the Selesnick-Burrus equations in the relatively simple 
case L = 1, M = 5, K = 1. There are 6 quadratic equations, from setting 

2m\ — 2to 2 , 

6m 2 + 2?7l4 — 8TO1TO3, 

2OTO3 — 2to@ + 12miTO5 — 3OTO2TO4, 
7OTO4 + 2to8 — I6TO1TO7 + 56TO2TO6 — II2TO3TO5, 
252m;? — 2mio + 20miTOg — 90m2TO8 + 24OTO3TO7 — 420to4TO6 
- to 3 + TO4TO2 

equal to zero, and similarly 4 additional linear equations: 

- 315 + 14496toi + 23912m 3 - 9310to 4 + 8to 7 - 196m 6 + 1904m 5 - 30184m 2 , 
2m 8 - 728to 6 + 9408to 5 - 51632m 4 + 141120m 3 - 185152m 2 + 91392toi - 2205, 
4m 9 - 17052to 6 + 247380m 5 - 1445010to 4 + 4105160to 3 - 5529048to 2 

+ 2784096mi - 72765, 
TO10 - 43407m 6 + 670320to 5 - 4070200m 4 + 11869200m 3 - 16288944m 2 

+ 8326080mi - 231525. 

In this small example, we can apply a "brute force" method to derive a solution. 
This is also essentially the method used by Selesnick and Burrus to handle the more 
difficult problems in their Region II. The lex Grobner basis for the whole system 
with TO10 > TO 9 > ••• > toi is in generic "Shape Lemma" QCLO], Chapter 2, 
§4) form. The last element is a univariate polynomial of degree 16 in m\. Using 
numerical root-finding, we find 6 approximate real roots: mi = .04470426799, 
1.233505559, 2.558981682, 4.441018318, 5.766494441, and 6.955295732. Then the 
other moments mj and filter coefficients h[i] can be determined by backsolving in 
the Grobner basis and using the equations (2.3). 

We can see a general feature of the Selesnick-Burrus equations here. Note that 
the 6 real roots form three pairs of the form r, 7 — r. In fact, for all K, L, M, the 
mapping 

(2.6) miH(L + M + K)-m± 

gives the effect of time reversal (that is, taking the original transfer function H(z) = 
E„=o h[n]z- n to the reversed H{z) = J2"=o h[N-l — n]z n ). It is not difficult to 
see that the whole Selesnick-Burrus system - (2.4a), (2.4b), and the linear equations 
expressing the higher moments in terms of the lower ones - is invariant under 
time reversal. Up to time reversal, there are 3 distinct real filters satisfying the 
Selesnick-Burrus conditions in this case. The plot in Figure 1. shows the square 
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magnitude response curves for the three filters. Note that two are apparently 
monotone decreasing, while one has a pronounced "ripple" in the "passband" . The 
filters with monotone square magnitude responses would be much more useful for 
actual low-pass filtering applications. 




Figure 1. 

The case we treated above: L = 1, M = 5, K — 1 is just within Selesnick and 
Burrus's Region 11 (sec §1). However, "brute-force" methods only work in very small 
cases in Region II! For instance, when L = it can be seen in several different ways 
that there are 2 M complex solutions of the Selesnick-Burrus equations. Thus solving 
the systems with L = becomes exponentially more complex as M increases. 

§3. A Solution Strategy in Region II 

In this section, we will present a strategy for solving the Selesnick-Burrus equa- 
tions in Region II that is much more efficient than "brute force" elimination as in 
Example (2.5). The idea is to exploit the special structure of the Selesnick-Burrus 
equations as much as possible. We will also report some results obtained by this 
strategy. 

First, following Selesnick and Burrus, we show how to reduce the number of 
variables from N— l = K + L + MtoM — L— 1 and obtain an equivalent system of 
equations that we will call the reduced Selesnick-Burrus system for a given collection 
of parameters K, L, M. The computations involved in these steps are minimal. 

The first part of this reduction is to use the simple observation that the last 
condition (1 + z^ 1 ) K \H{z) in Selesnick and Burrus' formulation implies that the 
moments mo,.. . , mjv-i already satisfy certain linear equations, and hence all of 
the equations can be expressed in terms of the moments in the column vector 
m = (too, • ■ • ,mL+M) tr ■ (As noted before, we will also normalize mo = 1.) 

To see how this works in detail, write H{z) = (l+z~ 1 )- ff P(z), let h be the column 
vector (h[0],h[l},... , h[N - I])* r , and \etp = (p[0],p[l], . . . ,p[N — I — K]) tr be the 
column vector of coefficients in P. Then we have an equation 

(3.1) h = Tp, 

where T is an N x (N - K) = (K + L + M + 1) x (L + M + 1) matrix whose rows and 
columns are shifted copies of the vector of binomial coefficients , j = 0, . . . , K. 
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By the definition (2.3) of the moments, we have 

(3.2) m = Qh = QTp, 

where Q is an (L + M + 1) x (K + L + M + 1) "Vandcrmonde-type" matrix, whose 
ith row is the vector of ith powers of the integers 0, 1, . . . , K + L + M. 
Combining (3.1) and (3.2), we have the equality 

h = T(QT)- 1 m 

Hence, we can express nik for k > L + M as 

(3.3) TO fe = (0, l fe , 2 fc , . . . , (L + M) fc )T(QT)- 1 ™ 

The second part of this reduction is to use some observations about the Selesnick- 
Burrus quadratic equations (2.4a) and (2.4b), and the afhne variety they define over 
the field C. It is well-known that there is nothing special about varieties defined by 
quadrics, but the Selcsnick-Burrus equations have a very particular form. First note 
that the quadratic Selesnick-Burrus polynomials do not depend on the parameter 
K. Let Jl,m be the ideal they generate in C[mi, . . . ,to 2 m]- In addition, we have 
the following observations. 

(3.4) Lemma. Let Vl,m = V(Jl,m) be the affine variety defined by the Selesnick- 
Burrus quadrics for a given pair of parameters L, M. 

a. The Selesnick-Burrus quadrics are homogeneous if we assign 

weight(mi) = i. 

b. Vl.m contains a rational normal curve passing through each of its points. 
c Vl.m is a smooth variety in C 2M of dimension M — L. 

Proof. All of these claims are easy consequences of the form of the quadrics. □ 

In fact, we can see much more about the variety Vl,m if look at another gener- 
ating set for the ideal that defines it. Before giving the general statement, we again 
take up the case K = 1, L = 1, M = 5 considered in Example (2.5). 

(3.5) Example. Recall the Selesnick-Burrus quadrics given in Example (2.5). If 
we compute a lex Grobner basis for J^ 5 with m w > m g > • • • > mi we find: 

G = {m.2 — mf, m 3 — mf, m 4 — m\ , 
m e — 6mim 5 + 5m®, m 8 — %m\m-j + H2mlm 5 — 105mi, 
mio - 10mim 9 + 240m?m 7 - 126m;? + 3780m^m 5 + 3675m} }, 

the Grobner basis G shows a very nice parametrization for V\^. If we let 

<p : C 4 -> C 10 

(t, a, b, c) i-> (t, t 2 , i 3 , i 4 , a, 6at - 5i 6 , 6, 8bt - 112a< 3 + 105£ 8 , c, 
Wet - 2406i 3 + 126a 2 - 3780ai 5 - 3675i 10 ) 
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The image of ip is precisely V1.5. 

We next indicate a connection between the Selesnick-Burrus systems and some 
classical topics in algebraic geometry. These observations are needed here only to 
verify that the hypotheses of [BEM] are satisfied for these systems and can be 
omitted if the reader is not familiar with these concepts. However, they motivated 
a large portion of our work on this problem. 

The related ideal 

J' = (m 2 — m\, m 3 — m\, m± — m\, m e — 677717775, m 8 — 877717777, m w — lOmimg) 

is equal to the ideal generated by the 2x2 minors of 

/ mi rri2 m 3 7774 m§ mg mio \ 
y 1 mi m 2 m 3 6m 5 87777 10mg J 

Hence, S — V(J') is an affinc 4-fold rational normal scroll (see, e.g. [H]) - the 
union of C 3 's spanned by related points on a rational normal curve of degree 4 and 
3 lines. Moreover, is the image of the scroll S under a certain upper-triangular 
automorphism a of C 10 . We also see that the projection of Vi t s into the coordinate 
subspace C 8 with coordinates mi, . . . , mg is itself a rational scroll of dimension 3. 
(It is only the quadratic term a 2 in the last coordinate that keeps V\$ from being 
a rational scroll itself.) 

Similar results hold for all the Vl,m- These observations imply that Vl,m is a 
unirational variety for all L, M. The additional linear equations define the affinc 
part of a 0-dimensional linear section of Vl,m- Because of this, the Selesnick- 
Burrus systems fall into the general context discussed in the paper [BEM], and we 
can use the main theorem there to eliminate variables using resultants (without 
using Grobner bases). We will use this approach in the following. 

Our next Lemma establishes an important common feature of all of the Selesnick- 
Burrus systems which we will exploit to define the reduced Selesnick-Burrus system 
for a given set of parameters K, L, M with M > L. 

(3.6) Lemma. Assume M > L. The Selesnick-Burrus quadrics (2.4a) and (2.4b) 
imply that m^ = m\ for all k, 1 < k < 2L + 2. 

Proof. The proof is by induction on L, the base case being L = 1. In that case, 
we have from (2.4a) with M = 1, 2, and m = 1: 

(3.7) 2m\ — 2m 2 , 6m 2 . + 2m 4 — 8mim 3 
From (2.4b) with L = 1: 

(3.8) -m 3 + mim 2 

The equation m 2 = m\ follows directly from the first equation in (3.7). Substituting 
in (3.8), we have 777,3 — m\. Then substituting in the second equation in (3.8), we 
have 7774 = m\. 

The induction step is similar. Assume we have shown that the quadrics (2.4a) 
with 1 < i < M and (2.4b) 1 < j < L imply m k = m\ for all 1 < k < 2L + 2. 
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Consider these quadrics, plus (2.4a) with i = M + 1 and (2.4b) with j = L + 1. 
By the induction hypothesis, we substitute m/, — m\ for 1 < k < 2L + 2. Then 
substituting into (2.4b) with j = L + 1, we have 



+ (-i) L+i (i- 



2L + 3J V 

2L + 2\ {2L + 'i 



2L + 3J \L + l 



This implies m2L+3 = m\ L+3 because, applying some standard binomial coefficient 
identities, 

Then, we substitute rrik — m\ for k = 1, . . . , 2L + 3 into (2.4a) with i = 2L + 4 to 
deduce m 2 L+4 = m^ L+4 . □ 



(3.9) Definition. The reduced Selesnick-Burrus system for given parameters 
K,L,M is the system of equations obtained from the full Selesnick-Burrus sys- 
tem of N — 1 = K + L + M equations in N — 1 variables mi, . . . tojv_i as follows. 

(1) First, substitute in the equations (2.4a) for i > L + 2, for all the moments 
toj. for k > L + M from equations (3.3) above. 

(2) Write mj = t and substitute m k = t k for all 1 < k < 2L + 2 in these 
equations. Also set m = 1. 

The result is a system of M — L — 1 equations in the M — L — 1 variables 



"^2L+3, ■ • • , niL+M- 



The quadrics (2.4a) with i < L + 1 and all of the quadrics (2.4b) are discarded since 
they have been used to derive the equations mk = t k . 

The Grobner basis computation we used in Example (2.5) and substitution of 
the parametrization of the variety defined by the Sclcsnick-Burrus quadrics does the 
same sort of elimination of variables as given in part 2 of the reduction described 
here (and more). Note that the linear equations we discussed above, for instance 
in Example (2.5), have been subsumed in the equations (3.3). We have eliminated 
the higher moments mk, k > L + M using them, so they do not appear explicitly in 
the reduced system. The parameter K enters only in the form of the T matrix in 
(3.3). Changing K changes the coefficients of the equations but not their Newton 
polytopes or the number of solutions (provided K > 1). 

It will be most useful to view the polynomials in the reduced system as polyno- 
mials in the moments TO2L+3, . . . , whose coefficients are polynomials in t. 
For the Region I cases considered by Selesnick and Burrus, these polynomials are 
linear in m 2 L+z 1 ■ ■ ■ , m-L+M, and this is what allows the use of purely linear algebra 
techniques to eliminate and obtain a univariate polynomial in t. 
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In fact the Region II cases are characterized by the fact that the reduced system 
still has non- linear terms in m 2 L+3, • • • , tul+m- The precise form of the reduced 
system is determined by "how far down into" Region II we are from the boundary. 
That is, for L sufficiently large, all the cases along the "diagonals" defined by 
M = 2L + q, for fixed q > 3 will have a similar shape. (There are also "special 
cases" along early portions of lower diagonals M = 2L + q with q > 5. These are 
different from the stable form because the nonlinear terms are different.) 

(3.10) Strategy. To study the Selesnick-Burrus equations for cases in Region II, 
we propose the following strategy. 

(1) Form the reduced system as in (3.9), and view it as a system of M — L — 1 
linear and quadratic equations in the M — L — 2 variables to 2 l+3, • ■ • , tul+m, 
with the variable t "hidden in the coefficients." 

(2) Use the linear equations in the reduced system to solve for a subset of the 
remaining higher moments in terms of the lower moments, and substitute 
into the quadratic equations. 

(3) Use an appropriate formulation for multipolynomial resultants to eliminate 
the remaining undetermined moments and produce a univariate polynomial 
in t. 

In order to compute examples, we have used several different resultant formula- 
tions. For instance, in §5 below, we will see that the cases with M = 2L + 3 can be 
handled by using the multipolynomial resultant of a general system of L + 1 homo- 
geneous linear equations and 1 homogeneous quadratic equation in L + 2 variables. 
This resultant is denoted by in [CLO], Chapter 3. 

Mixed sparse resultants (see [CE], [S]), Dixon (or Bczout) resultants (see [KSY]), 
and even the naive approach of iterated pairwise Sylvester resultants all work rea- 
sonably well on the smaller examples. Dixon resultants seem to be far superior 
for the larger cases. In almost all cases, some care is needed to eliminate extra- 
neous factors in the computed polynomial in t. One useful criterion here is the 
fact mentioned above in (2.6) that the Selesnick-Burrus system is invariant under 
time-reversal. Thus correct univariate polynomial in t must be invariant under 
t i ► (K + L + M) — t. This strategy is particularly well adapted for the problem 
of determining the number of complex solutions of the design equations as a func- 
tion of the design parameters K, L, M. In combination with numerical rootfinding 
methods, it can also serve as a template for a general solution method for the 
Selesnick-Burrus systems. We illustrate this below. 

To indicate the scale of the problems that this strategy allows us to solve, we 
provide the following table giving the degree of the univariate polynomial in t 
generating the elimination ideal of the Selesnick-Burrus system for given L, M. In 
most cases the computation was done with K = 1 for simplicity, but the degree 
will be the same for all K > 1. 

In this table, the entries along the diagonal M = 2L + 3 are the first within 
Region II; the Region I cases with M < 2L + 3 are not shown. For purposes of 
comparison, the entries for M < 7 were also reported by Selesnick and Burrus in 
[SB]. The entries with M > 8 and L > are new. Starred entries were computed 
by Robert Lewis of Fordham University, using his Fermat system and his routines 
for Dixon resultants. The blank entries are somewhat beyond the scope of current 
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software. On the other hand, many cases with M > 15 would also be tractable by 
these methods. 



M/L 





1 


2 


3 


4 


5 


5 


32 


16 










6 


64 


26 










7 


128 


48 


24 








8 


256 


78 


38 








9 


512 


152 


66 


32 






10 


1024 


278 


112 


50 






11 


2048 


512* 


192* 


86 


40 




12 


4096 


944* 


358* 


142 


62 




13 


8192 




572* 


240* 


106 


48 


14 


16384 




1020* 


402* 


174* 


74 



Figure 2. 



We will now present an outline of the resultant computation for the case K = 
2, L = 2, M = 10 and show how the methods described in [BEM] and [M] can be 
used to derive all the real solutions. The reduced Selesnick-Burrus system in this 
case is a system ofM — L — 1 = 7 equations in the 7 variables t — mi, and the nij, 
2 = 1,... , 12. We will begin by using the resultant to eliminate mj, j = 7, . . . ,12 
and yield a univariate polynomial in t satisfied by all the solutions. This is done by 
"hiding the variable t in the coefficients" of the system as described, for instance, 
in [M\. 

For simplicity, we will write mj = x, mg = y, mg = z, mio = u, mn = v, 
mi2 = w, and denote the jth equation by aj(x,y, z,u,v,w) = 0. The first three 
equations are 

= (Zi (x, y, z, u, v, w) = 7t s + y — 8tx 

= a 2 (x, y, z, u, v, w) = -84i 10 - u + lOtx - A5yt 2 + 120xt 3 

= a 3 (x, y, z, u, v, w) = 462i 12 + w - I2tv + 66t 2 u - 220zt 3 + A95yt 4 - 792t 5 x 

The remaining four equations are significantly more complicated and will be omitted 
here. (The complete computation is available as a Maple 8 worksheet from the 
author's homepage by downloading 

mathcs . holycross . edu/~little/SB2210 . mws 

To run this and other examples, the procedures in the file 

mathcs . holycross . edu/~little/CompFileLatest .map 

should also be downloaded.) 

The Dixon resultant computation proceeds as follows. We introduce a second 
set of variables X, Y, Z, U, V, W and compute the 7x7 determinant A whose jth 
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row is the transpose of 

/ aj(x,y,z,u,v,w) \ 

aj(X,y,z,u^v,w)—aj{x,y,z,u^v,w) 



X 


-x 


a,j(X,Y,z,u,v,w)- 


-cij(X 1 y,z,u,v,w) 


Y 


-v 


aj(X,Y,Z,u,v,w)- 


-aj{X,Y,z,u.v,w) 


Z 


-z 


aj{X.Y,Z,U,v,w)- 


-aj{X.Y,Z,u,v,w) 


U- 


-u 


aj(X,Y,Z,U,V,w)- 


-aj(X,Y,Z,U,v,w) 


V 


—v 


aj(X,Y,Z,U,V,W) 


- aj (X,Y,Z,U,V,w) 


W 


-w 



The expanded form of the determinant can be written as a matrix product A = R ■ 
M-C, where R is a 44-component row vector containing monomials in x, y, z, u, v, w, 
M is a 44 x 36 matrix whose entries are polynomials in t, and C is 36-component 
column vector whose entries are monomials in X, Y, Z, U, V, W. The rank of the 
matrix M in this case is 24. 

By the main result of [BEM], any 24 x 24 submatrix M' of M of rank 24 has 
determinant equal to a multiple of the resultant of the system. For a particular 
choice of maximal rank submatrix, we computed and factored the determinant 
yielding a reducible polynomial with one factor of degree 112 in t and other factors of 
smaller degrees. The factor of degree 112 is the resultant; the others are extraneous 
factors that depend on the choice of the submatrix M' . 

Using Maple's f solve routine, 12 approximate real roots were determined, t = 

.021826159039817- •• , 
1.14111245031295- •• , 
2.46849175059426 
4.77577862421111 ••• , 
5.42248255383217- •• , 
6.63285847397435 • • • 

and six additional roots obtained from these by time reversal - t *—> 14 — t (note 
that K + L + M = 2 + 2 + 10 = 14). In this computation, a 170 decimal digit 
floating-point number system was necessary to obtain accurate results. The use of 
the moment variables in the Selesnick-Burrus formulation simplifies the form of the 
equations immensely and makes the symbolic approach we have used feasible. But 
it also imposes a severe numerical conditioning penalty in return. 

To determine the other components of the solution, we use the form of the row 
monomial vector R in the the equation A = R ■ M ■ C above. The entries of R 
corresponding to the rows of the maximal rank 24 x 24 submatrix M' contain the 
six monomials x,y, z,u,v,w. Substituting each of the t values above in turn, the 
vector in the kernel of (M') tr with first component equal to 1 has 6 components 
equal to the x, y, z, u, v, w values in the corresponding solution of the system. We 
then determine the values of the filter coefficients from the moments from (3.2) and 
(3.3) above. 

The square magnitude responses of the 6 real filters found above are shown in 
Figure 3. Of these, four are apparently monotone decreasing, one has a maximum, 
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and one has a minimum and a maximum. The four monotone filters come from the 
t- values closest to the center value t = K+L ^+ M = 7. 



Timings for this computation are as follows (all done in Maple 8 on a SunBlade 
100 workstation with a 500 MHz UltraSPARC processor and 256MB of RAM, 
running Solaris). The symbolic part of the computation (the computation of the 
Dixon resultant and factoring the univariate polynomial) takes approximately 320 
seconds. (There is a certain amount of randomness built into the choice of the 
maximal rank submatrix M', however, and the time can vary depending on which 
submatrix is used.) The numerical part (the rootfinding steps) can be done quickly 
(i.e in less CPU time than the symbolic computation, even with the high-precision 
arithmetic) with an ad hoc "by-hand" search for the real roots in the interval [0, 7] 
and a fast iterative method like Newton-Raphson. (With the "brute-force" applica- 
tion of Maple's f solve command described above, and illustrated in the worksheet 
mentioned before, the numerical part of the computation takes much longer, of 
course - about 8200 seconds, including the plotting of the square magnitude re- 
sponse curves.) 

We have used similar numerical computations to solve the reduced system and 
determine the filter coefficients of the real solutions in many of the cases reported 
above in Figure 2. As is indicated by this example, we note that the degrees give 
only one measure of the complexity of these computations. 

In a companion paper [LL] , we discuss some properties of the filters obtained by 
these computations in more detail. In the next sections here we will focus instead 
on some of the patterns that seem to appear when the table in Figure 2 is examined 
carefully. 

§4. A Technical Interlude 

In this section we will prove a number of technical lemmas on the Smith normal 
form of certain matrices that appear when the linear equations in the reduced 




omega 



Figure 3. 
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Selesnick-Burrus systems (3.9) are reformulated in a particularly useful way. For 
simplicity, we will describe the general form of these matrices in this section in the 
abstract, so to speak; we will delay showing how the Selesnick-Burrus equations fit 
these patterns until §5 and §6. 

We will need the following notation. 

Notation. Let j, K, I denote nonnegative integers, and t an indeterminate. All 
vectors are infinite, indexed by the nonnegative integers, Z> . 

a. We will write A J for the vector of coefficients in the jth forward difference op- 
erator, each entry divided by j!, "padded" with additional zero entries on the 
right: 

^4(,- 1 ye ),(- 1 r.e 1 ),(- 1) -e 2 ),...,Q,o,...). 

The indices of the nonzero entries shown run from to j. 

b. Similarly, we will write Aj, for right shift by £ of the vector above, so the 
j, (— l) J '(o) occurs in position £, and zeroes appear in locations through I — 1. 

c. We will write 

The vector D 3 K can also be viewed as the padded vector of coefficients of a 
difference operator. 

d. We will write (i — t) 1 for the vector with entries 

((o-t)Mi-t)M2-t)V..) 

e. We will use the shorthand 

\j,£;K} = (D 3 K ,(i-t) 1 }, 

where (,) is the formal dot product on vectors indexed by Z> . Note that all 
of the vectors D 3 K we consider have only a finite number of nonzero terms, so 
convergence is automatic. The sum is the value at i = of the result of applying 
the operator D J K to the function of the discrete variable i given by (i — t) 1 . This 
is a polynomial of degree £ — j in t if I > j, and equals zero otherwise because 
all jth differences of a polynomial of degree < j in i vanish. 

f. An expression of the form [j, £; K] (a) will denote the value obtained by substi- 
tuting t — a in the polynomial [j, £; K] . 

(4.1) Lemma. The [j,£;K] polynomials have the following properties, 
a. ("reflection identity") Up to a sign, [j,£;K] is symmetric about t = 

\j,£;K](j + K-t) = (-iy+ e [j,£;K](t) 

We call t = the center value of [j, £; K] . 
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b. ("center value zero") If £ and j have opposite parity, then 

c. ("boost identity") [j,£;K] satisfies 

\j, £; if] (t - 1) = [j, £; if] (t) + (j + l)\j + l,£, K] (t). 



Proof. Part a follows from a direct computation. Because of the symmetry of the 
binomial coefficients in the A J e , D^ K is symmetric about up to the sign (— l)- 7 . 

Therefore we have 

[j, £■ K]((j +K)-t) = {D 3 K , (i ((j + if ) - t)Y) 

= { -iy {D i K ,(((j+K)-i)-ty) 
= (-iy+ ( (D j K ,(i-tY), 

= (-iy+%£;K}(t). 
Part b follows immediately from part a. 

Part c is shown by another calculation. In terms of the shift operator E, we have 
D k' = W(JTTy.( E + l ) K ^ E 80 



(j + l)[j + l,£;K](t) = (j + l)(D j +\ (i tf 



= ^((E + i) K (E-iy+\^-tY) 

= {D j K E,(i-t) i )-(D j K ,(i-tY) 
= \j,£;K](t-l)-[j,£;K](t).n 



The specific matrices that will appear in the analysis of the linear equations 
in the reduced Selesnick-Burrus systems have the following forms A(s,m;K) and 
A(s,m;K), for certain positive integers s,m depending on the flatness parameters 
L, M from the filter design problem. First we introduce the matrix A(s, m; K) = 
(4.2a) 

/ [2s -1,2s; if] [2s, 2s; if] ... [2s + m - 1, 2s; K] \ 

[2s -1,2s + 2; if] [2s, 2s + 2; if] ... [2s + m - 1, 2s + 2; if ] 

V[2s- 1, 2s + 2m; if] [2s + 1, 2s + 2m; if ] ... [2s + m - 1, 2s + 2m; if] J 

We will write <5(s, m; if) = det A(s, m; if). 
Similarly, A(s, m; if) = 



(4.26) 



/ [2s, 2s; if] [2s + 1,2s; if] ... [2s + m, 2s; if ] \ 

[2s, 2s + 2; if] [2s + 1,2s + 2; if] ... [2s + m, 2s + 2; if ] 

V [2s, 2s + 2m; if ] [2s + 1, 2s + 2m; if ] ... [2s + m, 2s + 2m; if ] / 
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We write 5{s,m;K) = det A(s , to; K) . 

For example, with s = 3, m = 1, and K = 2, the matrix A(3, 1; 2) is 

A(3 1-2)=( 21 ~ 6t 1 ^ 

K ' ' 7 v -56i 3 + 588t 2 -2212i + 2940 476 - 224t + 28i 2 / ' 

The entry in the second row and second column is [2s, 2s + 2; K] = [6, 8; 2}. 
The following observation will simplify our work considerably. 

Observation. Since [j, £; K] is zero if j > £, note that all the entries on the first 
row of the matrix A(s,m;K) except the first are zero. Expanding along the first 
row, we have 

5(s,m;K) =6(s + l,m- 1;K). 

Therefore, for our purposes it will suffice to study the S(s,m; K). 

Our main goal in the remainder of this section is to determine the Smith normal 
form of the matrices A(s,m; K) above, and hence to determine S(s,m; K). Recall 
that the Smith normal form of a square matrix A with entries in C[t] is the diagonal 
matrix obtained by doing elementary row and column operations. The diagonal 
entries satisfy the following property for all n < rank(A): the product of the first 
n diagonal entries is equal to the monic greatest common divisor of all the n x n 
minors of A. The properties of the Smith normal form follow from the standard 
theory of homomorphisms between modules over a PID such as C[t] (see for instance 

[Ja])- . , ' . . 

We introduce the following additional notation to facilitate working column by 
column in A(s, m; K). Note that the entries in A(s, to; K) all have the form [j, £; K] 
with 2s < £ < 2s + 2m, £ even. The entries in the first column have j = 2s — 1. The 
entries in the second have j = 2s, and so forth. We will write A 7 ' for the column in 
A = A(s, to; K) in which the entries are [j, £; K] for 2s < £ < 2s + 2m, £ even. 

Our first result shows that S(s, to; K) is symmetric about t = 2s + m - 1 + K ; U p to 
a sign. 

(4.3) Lemma. Let S(s, to; K) be as above, and let ca — 2s + to — 1 + K is the 

center value of the entries of the last column in A(s, to; K) ). Then 

8(s,m;K)(c-t) = ±6(s,m;K)(t). 

Proof. Consider the column A 2s + m - 1 -p f or each < p < m. The center value 
for the entries in this column is t = By Lemma (4.1), part a, we have the 

corresponding column in A(s, to; K)(c — t) equals 

A 2s+m-l- P ( c _ t j = A 2s+m-l-p^ c _ p j _ ^ _ p ^ 

= [—\^ 2s+m ^ l ^p A 2s+m ^ l ^ p (t — p) 

Then we apply the "boost identity" (Lemma (4.1), part c) repeatedly to deduce 
that A 2s+m ~ 1 ~ p (t-p) equals A 2s+m ^ 1 ^ p (t), plus a linear combination of the terms 
A 2s + m - x -P+i(t), for 1 < q < p. It follows that the the column in A 2s+m - 1 -P(c - 1) 
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is in the span of the columns in A>{t) with 2s + m — 1 — p < j < 2s + m — 1, and 
hence 5(s, m; K)(c — t) = ±<5(s, m; K){t). □ 

Some factors in 5(s,m;K) are immediately clear from Lemma (4.1), part b. If 
j is odd, then the center value root t = ^ + 2 K of the entries in the column A? is 
also a root of <5(s,m; K). Moreover, it will will follow from the next lemma that 

(t — i^-^j with e > 1 divides 6(s, m; K) in some cases. 

(4.4) Lemma. Let R= [2s - 1, 2s + m - 1] n N. If j G R is odd and j + 2p is also 
in R, then 

A' (^4^) E Span [A (t±lL±V) : f e Rj > j,f even] . 

PROOF. Note that i +K + 2 P is the center value of the entries in the column A> +2p . 
The proof is a kind of double induction argument descending induction on the odd 
j G R, and ascending induction on p > such that j + 2p G R. In the base case for 
the outer induction, j is the largest odd integer in R. In this case, necessarily, p = 0. 
But then t = ^ + 2 K is the center value root of the column A J , so the conclusion of 
the Lemma follows. Similarly, if j is any odd integer in R and p — we see that 

A> (*±£) = G SpanjV ( J ' + ^ +2p ) : j' G > j,f even}. 

For the inductive step, assume that the conclusion of the lemma holds for a given 
j,p, and also for all odd j > j and all q such that j + 2q G R. If j + 2(p + 1) G R, 
then we consider A> ( ttI£+liEtll\ gy the "boost identity" from Lemma (4.1), 



part c, for all even £, 2s < I < 2s + 2m, we have [j, £; K] = 

bA k\ (i±4±5) - (j + + K] a + K + + » 



Hence 



j + K + 2(p+l) \ = A . ( j + K + 2p \ j+1 n + K + 2(p+l) 



In the second term, j + 1 is even and > j, so we do not need to do anything further 
with that. We apply the inductive hypothesis to the first term: 

^ (*±4±^) € Sp«n{^ (^±4±^) : ,' G i*,j' > e.en}. 

The entries in the A> ^ J+J ^ +2p ^ appearing in the linear combination are the 
\j',£;K] ( 3+K 2 +2p y By the "boost identity" from Lemma (4.1) part c again, we 
havebV;^] (i±^) = 

j + ^ + 2(p+l)\ , ,., , 1U „ , 1 , + # + 2(f>+l)\ 



bV;^]( - ? + X+ 2 2(P+1) ) +(/ + 1)L/ + M; ^( 



20 



SOLVING THE SELESNICK-BURRUS FILTER DESIGN EQUATIONS 



Hence A 3 ' ( 3+K 2 +2p ) € 

Span [a 3 (j+* + 2(P+D) , (3 + K + 2<p + l)j j 

In the first vector in this set, j' > j is even and this term matches the conclusion. 
In the second vector, j' + 1 > j is odd. Moreover, since for suitable q, j + 2(p+ 1) = 
W + 1) + 2<7 = j + 2q e R, we may apply the induction hypothesis to conclude that 
the second vector is also in Span{A 3 ' ( ^+ k+2 (p +1 ) ^ • j' g ^ j' > j even}. □ 

The main consequence we will draw from this lemma is the following corollary 
giving information about the Smith normal form of A(s, to; K) and 6(s, to; K). 

(4.5) Corollary. Let p > 0, let 2s-l + 2p G and Zet t = 2s - 1 + 2 p+^ ; ^ e center 
vafote /or i/ie column X 2s - 1 + 2 p^ XTien i/ie ranA; of A(s 1 m\K) at this t is at most 
m — p (i.e. the rank drops by at least p + 1 atthist). Hence (2t — (2s — l + 2p + K)) 
divides the lastp+l entries on the diagonal of the Smith normal form of A(s,m; K), 
and (2t - (2s - 1 + 2p + K)Y +1 divides S(s, to; K). 

Proof. By standard properties of the Smith normal form, all the claims here 
follow from the statement about the rank of A(s,m;K) at t = 2s ^ 1 + 2 p+ k _ xhat 
statement follows directly from Lemma (4.4): At this t, the p+1 columns A 2s ~ 1+2q , 
< q < p are all in the span of the remaining columns of A(s, m;K). □ 



For future reference, we note that Lemma (4.3) (the symmetry of 5(s,m;K) 
about t = ca = 2s - 1 + m + K U p to sign) implies the existence of additional roots of 
S(s,m;K) greater than ca- 

The foregoing establishes lower bounds on the multiplicities of the roots of 
S(s,m;K) at the center value roots of the columns A 3 for odd j. We next show 
that there are also roots of 8(s, to; K) at the center value i-values of the columns 
A 3 for even j. 

(4.6) Lemma. Let 2s + 2p 6 R and consider the center value t = 2s+2 p+ k f or the 
column A 2s+2p . If j is odd and j < 2s + 2p, then 

A ,[2s + 2p + K\ n f „ [2s + 2p + K\ . „ „ , 1 

A 3 ( |- J £ Span I A 3 ( 1- J : j < j 1 <2s + 2p, j' even \ . 

Proof. The proof is similar to the proof of Lemma (4.4) except that now we 
will proceed by ascending induction on p, and descending induction on j such that 
j < 2s+2p. The base cases arep = 0, j = 2s — 1, and more generally, p arbitrary and 
j = 2s+2p-l. By the "boost identity" ((4.1) parte), [2«+2p-l, t; K] ( 2s+2 2 p+g ) = 

(4.7) [2-+2P-1, /; K] ( 2S + ^ + K - 1 )-(2s + 2 P )[2s + 2p, i;K]^ S + 2p + ^ 
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Next, apply the "reflection identity" ((4.1) part a) to the first term on the right. 
The center value for ^ 2 s+2 P -i ig 2s+2 P -i+k ^ SQ 

2 S + 2 P -l + K-( 2s + 2 ? + K -l) = g£±|±g. 
Hence, since £ is even and 2s + 2p — 1 is odd, 

(4.8, [2., + 2 P - 1, f; A] (*±f±* - l) - - P. + HP - 1. * A', (*±|!±* 
Combining (4.7) and (4.8) for all even £, 2s < £ < 2s + 2m, we have 

^2s+2p-l ^ + 2p + ^ £ ^ | A2s+2p ^ + 2p + ^ I 

So the conclusion of the Lemma holds in these cases. 

For the inductive step, assume that the conclusion of the Lemma holds for for 
all odd integers j between j + 2 and 2s + 2p with the current p, and for all p < 
p. Consider the entries [j,£,K] in AK By the "boost identity" ((4.1) part c), 

\j,£,K]{^±^.) = 

,<-A (^f^ - i) - (, + W + M,*] (^±f±^) . 



Hence 

^ + 2P + ^ E Svan L 

In the second vector on the right, j + 1 > j is even so this term matches the 
conclusion of the Lemma. In the first vector, 2s + 2 p+ k _ \ = 2s+2(p-i)+a: ^ g 
center value for the column A 2s+2t - p ~ 1 \ and j < 2s + 2(p — 1). By the induction 
hypothesis, Ai ^+2 {p -i)+k ^ e 



Span 



But for each entry in one of these A>' , we can apply the "boost identity" again: 

[)',<; if] (2£±2fclH£) = 



Hence 



Af ^ + 2(P -1) + ^ £ Span | A , ^ + 2 P + ^ ^ + 2p + ^| 

The first vector in the spanning set matches the conclusion of the Lemma since j' 
is even. In the second term, j' + 1 > j is odd. Hence that vector can be written as a 
linear combination as in the conclusion of the Lemma by the induction hypothesis. 
□ 



Here too, the main consequence we will draw from this lemma is a corollary 
giving information about the Smith normal form of A(s, to; K) and 5{s, to; K). 
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(4.9) Corollary. Let p > 0, let 2s + 2p e R, and let t = 2s + 2 r>+ K , the center 
value for the column A 2s+2p . Then the rank of A(s, to; K) at this t is at most m — p 
(i.e. the rank drops by at least p + 1 at this t). Hence (2t — (2s + 2p + K)) divides 
the lastp+1 entries on the diagonal of the Smith normal form of A(s,m;K), and 
(2t - {2s + 2p + K))p +1 divides S(s, to; K). 

Proof. As in the proof of Corollary (4.5), all the claims here follow from the 
statement about the rank of A(s,m;K) at t = 2s + 2 p+ k _ That statement follows 
directly from Lemma (4.6): At this t, the p + 1 columns A 2s ~ 1+2q , < q < p are 
all in the span of the remaining columns of A(s, m;K). □ 



As in the case of the center value zeroes from Corollary (4.5), Lemma (4.3) (the 
symmetry, up to a sign, of S(s, to; K) under 1 1— > ca — t, where ca = 2s — 1 + to + 
K) implies the existence of a second, symmetrically located collection of roots of 
5(s, to; K) greater than ca- We are now ready for the major result of this section. 

(4.10) Theorem. Let ca = 2s — 1 + m + K as above. The determinant S(s, m; K) 
can be written in the form: 

2s-l+K+2m 

(4.11) 6{s,m;K) = a ]"[ (2t - i)L ""'a A "" J+ 1 

i=2s-l+K 

for some constant a. In the Smith normal form of A(s, to; K), the (to + 1, to + 1) 
entry is (a constant times) the product Hi^s-i+if ~ *) (one factor for each 
root), the (to, to) entry is a divisor of this polynomial whose roots are the roots of 
5(s, to; K) of multiplicity > 2, and so forth. 




The \ca — i\ in the exponent ensures the symmetry of the exponents in this 
expansion about ca- To make this somewhat intricate statement more intelligible, 
before proceeding to the proof, we give a small example. Consider the 4x4 matrix 
A(2, 3; 2), which has the shape 



A(2, 3; 2) 



Here and in the rest of this article we will use the standing notational convention: 

Notation, [d] is shorthand for a polynomial of degree exactly d in t. 

For instance, the entry [3] in the first column is the polynomial [3, 6; 2] = 425 — 
420t + 150t 2 — 20t 3 . The entries marked are actual zeroes. 

According to the formula in the statement of the Theorem, the center value of 
the 4th column is t = ^f. where ca = 2s-1 + to + ^ = 2- 2-1 + 3 + 2 = 8. The 
set of roots is symmetric about t = 4. The "predicted" value for 5(2, 3; 2) is 



6(2, 3; 2) = a(2t - 5)(2i - 6)(2t - 7) 2 (2t - 8) 2 (2t - 9) 2 (2t - 10)(2t - 11) 
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for some constant a. Using the computer algebra system Maple, we find 

<5(2, 3; 2) = 672 (2t- 11) {t - 3) (t - 5) (2i-5) (2t-7) 2 (2t-9) 2 (t - 4) 2 , 
and the Smith normal form of A(2, 3; 2) is: 



z 1 

o 1 



\ 



P3 (t) 

Vo o o P7 (t)J 



where 



and 



PsO) =t 3 -12f 2 + ^-t-63= i(2t-7)(2i-8)(2t-9) 
4 8 



665 



p 7 (t)=t 7 -28i 6 + ^i 5 -2170t 4 + 



134449 , 77203 389415 51975 



t° 



16 



■t 



P7(i) is the monic polynomial with roots t — 5/2,3,7/2,4,9/2,5,11/2 (all multi- 
plicity 1). We now proceed to the proof of Theorem (4.10). 

Proof. It follows from Corollaries (4.5) and (4.9) that the product in equation 
(4.11) divides S(s, m; K). If we knew that S(s, m; K) had the form given in (4.11), 
then the claims about the Smith normal form of A(s, to; K) would also follow from 
these Corollaries. Hence, to prove the Theorem it suffices to prove that the degree 
of S(s, to; K) equals the degree of the product in (4.11) in t. To compute the degree 
of S(s, to; K), recall the form of the matrix A(s, m; K) given in (4.2a). We have 





( W 


[0] 












[3] 


[2] 


[1] 


[0] •■ 


: : ) 


A(s,m\K) = 


[5] 


[4] 


[3] 


[2] •• 







V [2m + 1] 


[2m] 


[2m - 1] 


[2m - 2] • • 


• [to + 1] / 



where, as earlier, [d] denotes a polynomial of degree d in t. The entries are actual 
zeroes. By examining the form of this matrix, it is not difficult to see that because 
of the zeroes above the main diagonal, every nonzero product of entries, one from 
each row and one from each column, has the same total degree as the product of 
the entries on the main diagonal: 



1 + 2 + --- + (to + 1) = 



(to + 1)(to + 2) 
2 

(m+l)(m+2) 



Hence the degree of S(s, to; K) is no larger than 

But on the other hand, we will see that the product in (4.11) also has degree 
(m+iKm+2) _ Hence it foUows tnat S(s,m;K) equals the product in (4.11). To 
compute the degree of (4.11), we consider the cases to even and to odd separately. 
Hm = 2q is even, then the central value of the last column of the matrix gives one 
of the central value roots. The sum of the multiplicities in (4.11) gives 



2(1 + 1 + 2 + 2 + • • • + q + q) + q + 1 = (q + l)(2q + 1) 



(to + l)(m + 2) 



2d 
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Similarly with to = 2q + 1 an odd number, the total degree is 



2(l + l + 2 + 2 + --- + q + q+(q+l))+q + l = (g+l)(2g + 3) 



(ro+l)(ro + 2) 
2 



which concludes the proof. 



□ 



By the Observation above concerning the A(s, m; K) matrices, we have a parallel 
formula for 6(s, to; K). 



(4.12) Corollary. Let c= 2s+m+K. The determinant 5(s, to; K) can be written 
in the form: 



2s-l+K+2m 



(4.13) 5(s,m;K)=a T\ (2t - ^L "" 1 ^"" J+ 1 



i=2s+l+K 




multiplicity > 2, and so forth. 

Proof. This follows directly from Theorem (4.10), using the relation 6(s, to; K) = 



Here is an example, showing 5(2, 3; 2) for comparison with 5(2, 3; 2) computed 
earlier. Using Maple, we have 



In this section we will discuss the Selesnick-Burrus systems for parameters L, M 
satisfying M = 2L + 3. In particular, we will prove the following theorem which 
explains one pattern that can be seen in the table given in Figure 2. 

(5.1) Theorem. In the cases M = 2L + 3, L > 0, (the "corners" in Region II 
boundary), for all K > 1, the univariate polynomial in t in the elimination ideal of 
the Selesnick-Burrus equations obtained via Strategy (3.10) has degree 8L + 8. 

Before giving the details of the proof, we outline the method we will use. Along 
the first diagonal in Region II, of the M — L — 1 = L + 2 equations in the 
reduced Selesnick-Burrus system, L + 1 are inhomogeneous linear equations in 
™2L+3, • • • , msL+3 whose coefficients are polynomials in t. We will begin by show- 
ing how the coefficient matrix of this linear part of the system can be rewritten as the 
matrix A(L + 2, L; K) as defined in §4, times a suitable invertible lower-triangular 
matrix. The last equation (from the flatness condition F {2M \0) = F < 4i+6 )(0) = 0) 
contains the non-linear term m\ L+3 , plus linear terms in to 2 l+3, . . . ,to 3 l +3 . To 



S(s + l,m-l;K). 



□ 



6(2, 3; 2) = 36 (t - 5) (2* -7) (2i-ll)(t-4) (-9 + 2t) 2 



which agrees with (4.13) for this s, to, K. 



§5. The M = 2L + 3 Diagonal 
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eliminate to a univariate polynomial in t, we will use a formula for the multivariable 
resultant Resi i ... t i i a from Proposition 5.4.4 of [Jo] (see also Exercise 10 of Chapter 
3, §3 in [CLO]). (This formula may be proved by the basic approach of solving the 
linear equations for rri2L+3, ■ ■ ■ , ^3L+3 in terms of t by Cramer's Rule, then substi- 
tuting into the last equation to obtain a univariate polynomial in t.) We will need 
to keep careful track of the factorizations of S(L + 2, L; K) = det A(L + 2, L; K) 
from Theorem (4.10). The steps in this outline will be accomplished in a series of 
Lemmas. 



(5.2) Lemma. The L + l linear equations in the reduced Selesnick-Burrus system 
with M = 2L + 3 can be rewritten in the form 



A(L + 2,L;K) ■ £ ■ m r = b, 



where A(L+2, L; K) is the matrix defined in (4-2a), C is a constant lower-triangular 
matrix with diagonal entries equal to 1, m r — (m 2 L+3,--- ,mzL+z) tr , and b = 
([2L + 4],[2L + 6],... ,[4L + 4]) t '\ 

Proof. Recall the form of the Selesnick-Burrus quadrics from (2.4a): 

(5.3) = (^)mj + 2 g (-1)'+Wn*_, 

The linear equations in the reduced Selesnick-Burrus system come from these for 
j = L + 2, . . . , 2L + 2, via the reduction process described in (3.9). We begin by 
rearranging these equations to the following form by separating the terms involv- 
ing the variables m 2 L+3, ■ ■ ■ ,^3L+3 from those depending on the higher moments 
m 3L+4 , ... , m 4L+4 . We have 

(5.4) WiVi T(QT)- 1 m + W 2 V 2 T(QT)- 1 m = 9, 



where 



(1) the matrix W\ comes from the coefficients of the rrik, 2L + 3 < k < 3L + 3 
in (5.3): 



Wi 



i 2L 3 + y 3 



( 2 V 4 ) 





_r2L+y 



( 2L o +6 ) 



(2) the matrix W 2 comes from the coefficients of the nik, 3L + 4<fc<4L + 4 
in (5.3): 



••• 

w 2 - (-l) z ' 
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(3) the matrices V\, V 2 are Vandermonde-type matrices: 

/0 l 2i + 3 ••• (3L + 3 + K) 2L+3 \ 



and 



V 2 = 



\0 1 3L + 3 



/0 1 3L + 4 



(3L + 3 + K) 3L + 3 ) 



(3L + 3 + K) 3L+i " 



(4) 
(5) 



(6) 



(1,*, 



,t 2L+2 ,m 2L+3 ,. 



{3L + 3 + K) 



4L+4 



,m 3L+3 ) tr 

b' has the same form as b in the statement of the Lemma but is not the 
entire vector of t terms. (There are also terms depending only on t that 
come from the matrix product {W-iYi + W 2 V 2 )T(QT)- 1 m.) 
the matrices Q and T are as in the discussion leading up to (3.3). 

Since the first 2L + 3 entries of m depend only on t, the coefficients of m 2 L+3 
through TO3L+3 in our equations come from the product 



(W1V1 + W 2 V 2 ) -T-fhr, 

where T is the submatrix of T(QT)~ 1 containing all the entries from the last L + 1 
columns. The other terms in the product (W1V1 + W 2 V 2 ) ■ T{QT)^ 1 ■ rfi containing 
only powers of t go into the vector b, and (5.4) becomes 

(5.5) (WiVi +W 2 V 2 ) -T -rh r = b 



The fact that establishes the connection between these equations and the matri- 
ces A(s,m; K) considered in §4 is the following observation. In the matrix T, the 
final column is the vector Z?|^ +3 as in the Notation at the start of §4, written as a 
column. This follows if we think of the columns of (QT)^ 1 as operators acting on 
the rows of QT, thought of as power functions of a discrete variable. Similarly, the 
next-to-last column of T(QT)^ 1 is a linear combination D^ +2 + aD^ +3 for some 
constant a, and so on. In general, we have 

(5.6) T = (D 2 K L+3 \D 2 K L+4 \ ■ ■ ■ \D 3 K L + 3 ) ■ C 

for a lower-triangular square (L + 1) x (L + 1) matrix C with diagonal entries 1. 

To finish the proof of the Lemma, we substitute (5.6) into (5.5) and rearrange 
the terms again: 

(WW + W 2 V 2 ) (D 2 K L+3 \D 2 K L+4 \ ■ ■ ■ \Df+ 3 ) -C-rhr + b. 

We have 

Wi := V, (D 2 K L + 3 \D 2 K L +*\ ■ ■ ■ \Df+ 3 ) = (D^% 
for 2L + 3 < j < 3L + 3 and 2L + 3 < £ < 3L + 3., and 

U 2 := V 2 (D 2 K L+3 \D 2 ^\ ■ ■ ■ \Df +3 ) = ( D y e ), 
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for 2L + 3 < j < 3L + 3 and 3L + 4 < I < AL + A. 
Consider the (/, J) entry of the product 

(WWi + W 2 V 2 ) (Dl L +Z\Dl L+i \ ■ ■ ■ \D 3 K L + 3 ) , 

which is the dot product of the Jth row of W\ with the Jth column of U\, plus 
the dot product of the Jth row of W 2 with the Jth column of U 2 ■ The form of the 
entries in Wi and W 2 on the Jth row is (-1)« ( 2L + 2 ( I+1 ^t q for q from 21 - 1 down 
to 0. Hence this sum of dot products equals 

21-1 



]T (-l) q ( 2L + 2(7 + 1} ) t«D J K i 2L+2< > I+1 ^ = D J K {i- t )2i+2(/+l) 

g=0 \ 1 / 

= [J,2L + 2(I+1);K], 



using the notation introduced in §4. As / runs from 1 to L + 1 and J runs from 
2L + 3 to 3L + 3, we see that these entries form the matrix A(L + 2, L: K ) as claimed. 
□ 



For a general system of L+l linear homogeneous equations and one homogeneous 
quadratic equation in L + 2 variables, if the linear equations are written as Ax = 0, 
and the quadratic equation is Q(x) — 0, then by the result from Proposition 5.4.4 
of [Jo] mentioned before, the multivariable resultant Resi t ... t i t2 equals 

(5.7) Q{8 U -h, £3, • • • , (-1) L+1 <5l +2 ) 

where Si — detAi, and Ai is the [L + 1) x (L + 1) submatrix of A obtained by 
deleting column /. 

We apply this to our reduced Selesnick-Burrus system. Write the augmented 
matrix of the linear equations as 

A = (A(L + 2, L, K) ■ C\ - b) 

where C is the lower triangular matrix and b is the column vector ( [2L + 4] , [2L + 
6], . . . , [AL + A]) tr from Lemma (5.2). Our next Lemma shows that the determi- 
nants of the minors of A have a common factor of degree (2). To prepare for this 
statement, we introduce the following notation. Let S be the product of the first L 
diagnonal entries (elementary divisors) in the Smith normal form of A(L + 2, L; K): 

AL + l+K 

nL-\ZL + 3 + K-i\ 1 
(2t-*)L— ^ L J 

i=2L+5+K 

(There is one factor in this product for each of the roots of multiplicity > 2 of 
S(L + 2, L; K), and the exponents are each 1 less than the corresponding exponents 
in 5(L + 2,L;K).) 
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(5.9) Lemma. Let A be as in (5.6) and Si be the ith minor of A as above. If 
1 < i < L + 1, then 

5i = [AL + 3 + i] ■ 6 

where [AL + 3 + i] is some polynomial in t of degree AL + 3 + i, and S is the product 
from (5.8). Ifi = L + 2, then 

AL+3+K 

ni L- \3L + 3+K-i\ I , 
(2i - *)L^ L J+ 1 . = [2L + 1] • S 

i=2L+3+K 

Proof. We begin by computing the minor Al+2- Since £ is a constant lower- 
triangular matrix with diagonal entries equal to 1, Al+2 = 5(L + 2,L;K) = 
det^4(i + 2,L;K). We use Theorem (4.10) to compute this. We have ca — 
3L + 3 + K and 

4L+3+K 

5 L+2 = 8(L + 2,L;K) = a ( 2t ~ i) ^^^ J+1 

i=2L+3+K 

for some constant a. By the properties of the Smith normal form, we know that 
at a root t = t of multiplicity r, the rank of A(L + 2 1 L;K) is L + 1 — r, so every 
(L + 2 — r) x (L + 2 — r) submatrix of A(L + 2, L; K) will have zero determinant 
at t = to- 

Now consider the other minors A, for 1 < i < L+l, and expand the determinant 
along the column containing the entries from the vector b. Each term in this 
expansion is the product of an entry from b times the determinant of an L x L 
submatrix of A(L + 2, L; K) ■ C Hence by the statement at the end of the last 
paragraph, 8i is divisible by <5. The remaining factor in Ai comes by examining the 
degrees of the entries of the matrix as in the proof of Theorem (4.10). Note that if 
L = 1, the starting value of the index i is greater than the final value. In that case 
5 = 1. In all other cases, the degree of 5 is (2) (see the proof of Theorem (4.10)). 
□ 

We now consider the quadratic polynomial in the reduced Selesnick-Burrus sys- 
tem. Let z be a homogenizing variable. Then the homogenized version of Q, the 
equation from F( 4L+6 )(0) = 0, has the form 

(5.10) [0}m 2 2L+3 + [2L + 3}m 2L +3Z + ■■■ + [£ + 3}m 3L+3 z + [AL + 6]z 2 

We analyze the result of substituting the (— l)* +1 <5i into this polynomial as in (5.7). 

(5.11) Lemma. The resultant of our system has the form 

[8L + 8]S 2 , 

where [8L + 8] denotes a polynomial of degree 8L + 8 in t, and S is the product from 
(5.8). 
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Proof. To obtain the resultant of our equations to eliminate m 2 L+3, ■ ■ ■ ,m3L+3, 
we substitute 

m 2 L+3 = Si 



m 3 L+3 = Sl+i 

Z = S L+2 

into (5.10) (following equation (5.7) above), and use Lemma (5.9). We obtain the 
following expression for the resultant: 

[0]([4L + A]5f + [2L + 3] ([41 + 4]5)([2L + 1]S) + ■■■ + 
(5.12) [L + 3]([5L + 4]J)([2L + 1]5) + [4L + 6]([2L + 2}5) 2 

= [8L + 8}8 2 . 



□ 

The factor of degree 8L+8 is the univariate polynomial in t that we want, and this 
concludes the proof of Theorem (5.1). The other factor in (5.12) is extraneous in the 
sense that the t with S(t) = do not give solutions of the whole Selesnick-Burrus 
system. In fact, it can be seen that the linear equations in the reduced system 
are inconsistent for those t. In algebraic geometric terms, the resultant of the 
homogenized system contains information about all the solutions of the equations 
in projective space, including solutions "at infinity" . The common factor 5 2 gives 
solutions at infinity, and the degree in t of the full polynomial in (5.11) is the 
degree of the projective closure of the affine variety defined by the Selesnick-Burrus 
quadrics - the deformed rational scroll as in the discussion given in Example (3.5) 
in the case L = 1,M = 5. In that case there are no solutions at infinity (since 
5 = 1). However for L > 2, there are always such solutions. For example with 
L = 2, there are 24 solutions of the Selesnick-Burrus system for all K > 1, but the 
degree of the variety defined by the quadrics is 26. The factor 5 2 = [(I)] = [2] 
accounts for the difference. Similarly, with L — 3, there are 32 solutions of the 
Selesnick-Burrus equations for all K > 1, but the degree of the variety defined by 
the quadrics is 38. Again, the factor S 2 = [( 2 )] = [6] accounts for the difference. 

In the companion article [LL] , we will give more details on the structure of the 
filters corresponding to the 8L + 8 solutions of the Selesnick-Burrus equations for 
small L. For example, rather extensive calculations suggest the following conjec- 
tures. 



(5.12) Conjectures. Consider the Selesnick-Burrus equations with M = 2L + 3. 
and K > 1. 

(1) The polynomial of degree 8L + 8 is irreducible over Q, hence has 8L+8 
distinct solutions in C. 

(2) Of the 8L + 8 solutions, 2(L + 2) are real (yielding L + 2 different filters 
because of the invariance under time reversal). 

(3) Exactly four of these (2 different filters), those with t = m\ closest to 
the center value K + L + M ; yield monotone decreasing square magnitude re- 
sponse. 
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(4) The other solutions correspond to filters with progressively greater oscillation 
and greater maximum "passband ripple" as the distance from t = mi to 
K + L + M increases. 

The beginnings of this pattern can be seen in Example (3.5), which gives the 
case L = 1, M = 5. 

§6. The M = 2L + 4 Diagonal 

In this section we will discuss the Selesnick-Burrus systems with M — 2L + 4, 
the second diagonal in Region II in the table given in Figure 2. Our goal is to prove 
a result parallel to Theorem (5.1) giving the degree of the univariate polynomial in 
t whose roots give the different solutions. 

(6.1) Theorem. In the cases M = 2L + A, L > 1, for all K > 1, the univariate 
polynomial in t in the elimination ideal of the Selesnick-Burrus equations obtained 
via Strategy (3.10) has degree 12L + 14. 

Our proof will follow the same pattern as the proof of Theorem (5.1). First, we 
analyze the form of the equations in these cases. We rewrite the linear equations 
in a suitable form making use of the results of §4. Then the univariate polynomial 
is obtained via an elimination of variables tailored to the form of these equations. 

We begin by noting that the reduced Selesnick-Burrus system in these cases 
has the following form. The first L + 1 equations (from the flatness conditions 
F( 2i+4 )(0) = • • • = F( 4i+4 )(0) arc linear in the L + 2 variables m 2L +3, ■ ■ ■ ,m 3L+4 . 
The remaining two equations have nonlinear terms. The condition _F( 4L + 6 )(0) = 
gives a reduced equation containing ml L+3 , plus linear terms in all the variables. 
(This is the same as the last equation in the M = 2L + 3 cases.) In addition, the 
condition F< 4i + 8 ) (0) = gives a reduced equation containing m\ L+i , m 2 L+3'm2L+5, 
terms, plus linear terms. Following the strategy (3.10), we solve the linear equations 
for L + 1 of the variables in terms of the others, substitute into the quadrics, then 
compute the Sylvester resultant of the 2 quadrics. (Our approach here is closely 
related to one way to derive the multipolynomial resultant for a system of L + 1 
homogeneous linear and 2 homogeneous quadratic equations in L + 3 variables: 
fiesi,... ,1,2,2, but it seems to be easier in this case to use an ad hoc approach.) 

We begin with the following Lemma describing the linear equations, Since the 
precise statement involves some new quantities, we will sketch the derivation first, 
then give the formulation of the Lemma we will use. First, an argument exactly 
like the proof of Lemma (5.2) shows that the linear equations can be rewritten in 
the form 

A ■ C ■ m r = b 
where A is the (L + 1) x (L + 2) matrix: 

/ [2L + 3, 2L + 4; K] [2L + 4, 2L + 4; K] ■■■ [3L + 4, 2L + 4; K] \ 
[2L + 3, 2L + 6; K] [2L + 4, 2L + 6; K\ ■■■ [3L + 4, 2L + 6; K\ 

\[2L + 3,4i + 4;if] [2L + 4, 4L + 4; if] ••• [3L + 4, 4L + 4; K] / 

£ is a certain lower-triangular constant matrix with l's on the main diagonal, 
m r = (to 2 l+4, • • • , m 3L +4) tr , and b = ([2L + 4], [2L + 6], . . . , [4L + 4]) tr . We will 
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write {2L + 3, 2L + 2i + 2; K} for the entry in column 1 and row i of the matrix 
A ■ £ (a certain linear combination of the entries on row i of the matrix A) . After 
we subtract all terms involving TO2L+3 to the right-hand sides of the equations, we 
obtain the following result, because the submatrix of A consisting of all entries in 
the last L + 1 columns is precisely the matrix A(L + 2, L; K) from (4.2b). 

(6.2) Lemma. Using the notation introduced above, the L + l linear equations in 
the reduced Selesnick-Burrus system with M = 2L + 4 can be rewritten in the form 

A(L + 2, L; K) ■ C ■ m r = V, 

where V = 

({2L + 4] - {2L + 3, 2L + 4; K}m 2L+3 , . . . , [AL + 4] - {2L + 3, 4L + 4; K}m 2L+3 ) tr . 



We can solve the system A(L + 2, L; K) ■ C ■ m r = b' for the moments in m r using 
Cramer's Rule. For 1 < i < L + 1, this gives 

(6.3) TO2L+3+J 



S(L + 2,L;K) 

where Ai is the matrix obtained from A(L + 2, L; K) ■ C by replacing column i with 
the vector b'. Next, we consider what happens when we substitute from (6.3) into 
the first nonlinear equation (from F ( 4i+6 )(0) = 0). We will show that the result is 
an equation of the form 

(6.4) [0}m 2 2L+3 + [2L + 3]m 2 L +3 + [AL + 6] = 

(in other words, the denominators from (6.3) cancel with terms in the numerators 
in this equation). The situation that produces this cancellation is described in the 
following general lemma. 

(6.5) Lemma. Consider a system of equations of the form 

an{t)xi + a 12 (t)x 2 H h a ln x n = n(t) 

a 2 \{t)xi + a 22 (t)x 2 H h a 2n x n = r 2 (t) 

a„-i,i(t)xi + a n - li2 (t)x 2 H h a n - lin x n = r„_i(i) 

a n i{t)xi + a n2 (t)x 2 H h a nn x n = r n (t) + cx\, 

where a,ij(t) and n(t) are in C[t}. Let A = (aij(t)) be the full n x n matrix of 
coefficients of the linear terms, and let A' = (aij(t)), 1 < i < n — 1, 2 < j < n be 
the matrix of coefficients of x 2 , . . . ,x n in the first n—l equations. Assume, up to a 
constant factor, det A' is the product of the of the first n—l diagonal entries of the 
Smith normal form of A. Then solving for x 2 , . . . ,x n from the first n—l equations 
by Cramer's Rule and substituting into the last equation produces an equation of 
the form 

cx\ + B(t)xi + C(t) = 0, 
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where B(t),C(t) G C [t] . 

Proof. To make the connection between A and A' clearer, we note that A' = A n \ 
(submatrix obtained by deleting row n and column 1. We will number the rows 
in A' by indices 1 through n — 1 and the columns by indices 2 through n in the 
following. As described above for the Selesnick-Burrus equations, take the first 
n — 1 equations, subtract the x\ terms to the right sides, and apply Cramer's Rule 
to solve for x 2 , . . . , x n in terms of x\, yielding: 

dct A 1 - 



det A' 1 

for 2 < j < n, where A'j is the matrix obtained from A' by replacing the jth column 
(recall, this means the column containing the the oijj(i), 1 < i < n — 1 for this j) 
with the vector 

(6.6) (n(t) -a n (t) Xl ,... ,r„_i(t) - a n - 1A {t) Xl ) tr . 

If we expand det A'j along the column (6.6) in each case we obtain an expression 

n-l 

det 4 = det A njXl + ^2(-l) l+J n(t) det A' tj 

%=i 

where A n j is the submatrix of A obtained by deleting row n and column j, and A^ 
is a minor of A' (which is also a submatrix of A obtained by deleting two rows and 
two columns). Substitute for Xi in the last equation in the system and rearrange, 
taking all the rj(t) terms to the right hand side. The coefficient of rj(t) is 1/detA' 
times (— l) n+ i dct Aj\. Hence we obtain 

— (^pa nj ■ (-l^det^-j x x = cx\ + (j2(-lT+J dct A 3irj (t) 

Up to a sign, the coefficient of x\ is 1/ dct A' times the determinant of A, expanded 
along the nth row. So this can be rewritten as 



( _ 1)n _idetA =cx2 
y ' det A 1 



By hypothesis, detA' divides detA and all of the det Aji, which finishes the proof. 
□ 

Now, we must show that the linear equations in the Sclcsnick-Burrus system sat- 
isfy the hypotheses of the Lemma. But this follows from the determinant formulas 
from §4. In our case, x\ = rri2L+3, and x 2 , ■ ■ ■ ,x n are TO2L+4, • • • , tusl+4- Contin- 
uing from Lemma (6.2), A is the matrix A(L + 2, L + l; K) (times a lower triangular 
factor of determinant 1), and A' is A(L + 2, L; K) (times another lower triangular 
factor of determinant 1). Hence by Theorem (4.10) we have ca — 2s + m— 1 + K = 
3L + 4 + K, and 

4L+5+K 

T— r 1 L + l-\3L+4+K-i\ , , , 

detA = 6(L + 2,L+l;K) = a J] (2* — i)L s J+ 1 

i=2L+3+K 
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for some constant a. Similarly by Corollary (4.12), we have c = 3L + 4 + K and 

4L+3+K 

detA' = S(L + 2,L;.fi:) = a' ][ (2t-i)L ^ L J +1 , 

i=2L+5+if 

for some constant a'. Because of the L — 1 in the exponent, each factor in det A' 
occurs with multiplicity one less than in det A. Hence det A' is precisely the product 
of the first L diagonal entries of the Smith normal form of the (L + 1) x (L + 1) 
matrix A, or equivalently, ^ is a polyomial whose roots are all the roots of 
det A = 0, but all with multiplicity 1. Hence the conclusion of Lemma (6.5) holds, 
and we obtain an equation of the form (6.4). 

Because of similar cancellations, the final nonlinear equation (from _F( 4L+8 '(0) = 
0) has the form 







™ 2 I 4 ^ + 4 ] [61* + 7] 

(6-6) [2}mt L+3 + + j^j = 

after we substitute for rri2L+4, ■ ■ ■ , "^3L+4 from (6.3). The polynomial of degree 
2L — 1 in the denominators is the same in both terms after the first, and equals 
the last diagonal entry in the Smith normal form of A(L + 2, L; K) (the reduced 
polynomial of the determinant 5(L + 2, L; K)). 

The final step is to eliminate t from the two equations (6.4) and (6.6). For this, 
we use the determinant form of the Sylvester resultant (see [CLO] ) of two quadratic 
polynomials, after clearing the denominators in (6.6). We have 



Res = dot 



/ [0] [2L + 3] [4L + 6] \ 

[0] [2L + 3] [4i + 6] 

[2L + 1] [4L + 4] [6L + 7] 

V [2L + 1] [4L + 4] [6L + 7]/ 

= [12L+ 14] 

This concludes the proof of Theorem (6.1). 

Computation of these polynomials for a number of L and K suggests that the 
polynomial of degree [12L + 14] is always irreducible over Q, hence has distinct 
roots in C. But we do not have a proof of this fact. The filters obtained in this 
case arc considered in [LL]. 

The strategy from (3.10) that we have used here and in §4 can also be used to 
analyze the lower diagonals M = 2L + q, q > 5 in Region II. For instance, for q = 5, 
solving the linear part of the reduced Selesnick-Burrus system and substituting 
into the remaining equations leads to a system of 3 quadrics in 2 variables (or 3 
homogeneous variables). Explicit dctcrminantal formulas for the multipolynomial 
resultant Res 2 ^,2 (sec [CLO], Chapter 3, §2) can be applied, and it can be seen 
that for L > 2, the degree of the univariate polynomial in t is 20L + 26. We will 
not present the details of that case here. 

However, the resultants needed to eliminate variables in the final, nonlinear 
system get progressively harder to analyze as q increases. Unfortunately, the Dixon 
resultants leading to the most efficient computations tend to have many extraneous 
factors that must be accounted for. As a result, they are less convenient for the 
type of analysis done here. 
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